United States Patent and Trademark Ofhce 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OtBce 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



10/016,941 



12/13/2001 



55648 7590 05/13/2009 

KEVIN L. RUSSELL 

CHERNOFF, VILHAUER, MCCLUNG & STENZEL LLP 

1600 ODSTOWER 

601 SW SECOND AVENUE 

PORTLAND, OR 97204 



SLA1045 (7146.0121) 



PILLAI.NAMITHA 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



KJtSiVrXS nvrliyjts OUff Iff fcff Jr 


Application No. 

10/016,941 


Applicant(s) 

ERRICO ET AL. 


Examiner 
NAMITHA PILLAI 


Art Unit 

2173 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 29 January 2009 . 
2a )□ This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-7.9-34.36.37.39-56 and 58-87 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) K Claim(s) 87 is/are allowed. 

6) |EI Claim(s) 1-7.9-20.22-34.36.37.39-47.49-56.58-68 and 70-86 is/are rejected. 

7) |EI Claim(s) 21.48 and 69 is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/IVIail Date. 

3) ^ Information Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date 4/9/09 . 6) □ Other: . 



PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 14 



Application/Control Number: 10/016,941 Page 2 

Art Unit: 2173 

DETAILED ACTION 
Response to Amendment 

1 . The Examiner acl^nowledges Applicant's submission on 1/29/09 including the 
addition of new claim 87. Claims 1-7, 9-20, 22-34, 36, 37, 39-47, 49-56, 58-68 and 70- 
86 are rejected. Claims 21, 48 and 69 are objected to. Claim 87 is allowed. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1 -7, 9-20, 22-34, 36, 37, 39-47, 49-56, 58-68 and 70-86 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent No. 6,219,837 B1 (Yeo 
et al), herein referred to as Yeo, U. S. Patent No. 6,678,635 (Tovinkere et al), herein 
referred to as Tovinkere and U. S. Patent No. 5,717,879 (Moran et al.), herein referred 
to as Moran. 

Referring to claim 1, Yeo discloses a method of presenting information regarding 
a video comprising a plurality of frames (column 1, lines 6-10). Yeo discloses 
summarizing a video, the summarization comprising a plurality of segments of the video 
(column 1, lines 6-10). Yeo discloses that each of the segments includes a plurality of 
sequential frames of the video (Figure 1 and column 1, lines 6-10). The segments 
represent sequential frames of past and further video data associated with the currently 
playing video. Yeo discloses displaying the summarization in a first portion of a display 



Application/Control Number: 10/016,941 Page 3 

Art Unit: 2173 

(Figure 1 ). Yeo discloses sequentially indicating the relative location of each of the 
plurality of segments within the summarization relative to at least one other of the 
segments, each of the plurality of segments represented by a bounded spatial region 
(Figure 7 and column 5, lines 13-21). The bounded region that is defined indicates 
video data as frames. This bounded region indicates the temporal relationship of the 
frames of the video with regions that visually indicate differences in the video data. Yeo 
discloses using a first visual indication and using a second visual indication different 
from said first visual indication to indicate different video segments (Figure 7). Yeo 
discloses receiving from the user, by interaction the graphical user interface, a selection 
of one of the plurality of segments (column 1 , lines 48-52). Yeo discloses in response 
to the selection, presenting a selected one of the plurality of segments and not 
presenting at least one other of the plurality of segments (column 1 , lines 48-57). 
Selecting one segment results in the presentation of that video segment as opposed to 
another video segment. 

Yeo does not disclose an event characterized by a semantic event that includes 
a sports play. Tovinkere discloses detecting an event in a video characterized by a 
semantic event that includes a sports play (column 7, lines 40-43). It would have been 
obvious to one skilled in the art at the time of the invention to learn from Tovinkere 
event characterized by a semantic event that includes a sports play. Tovinkere 
discloses that identifying the semantic event of a sports play ensures that the user has 
access to the most desired data within large amounts of video data. This provides 
motivation for Tovinkere to learn from Yeo. Therefore one skilled in the art at the time 
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of the invention would have been motivated to learn from Tovinkere event characterized 
by a semantic event that includes a sports play. 

Yeo and Tovinkere do not disclose displaying a graphical user interface on a 
second portion of the display with a plurality of segments. Moran discloses a graphical 
user interface with a plurality of segments (Figure 1 1 ). The plurality of segments has 
different visual indications that disclose relative location for events within a video (Figure 
11). It would have been obvious to one skilled in the art at the time of the invention to 
learn from Moran displaying a graphical user interface on a second portion of the 
display with a plurality of segments. Yeo discloses a timeline structure with a plurality of 
segments that visually indicate the different within the segments of the video data. The 
displaying of this timeline structure would allow the user to determine different types of 
events with the video data and access these events more easily and efficiently. This 
provides motivation for Yeo and Tovinkere to learn from Moran. Therefore one skilled in 
the art at the time of the invention would have been motivated to learn from Moran 
displaying a graphical user interface on a second portion of the display with a plurality of 
segments. 

Referring to claim 2, Yeo and Tovinkere disclose that the first and second 
semantic characterizations of a sports play temporally overlap in the summarization 
(Tovinkere, column 5, lines 50-60). 

Referring to claim 3, Yeo discloses that the graphical user interface includes a 
generally rectangular region where each of the plurality of segments is indicated within 
the generally rectangular region (Figure 1). 
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Referring to claim 4, Yeo discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 7). 

Referring to claim 5, Yeo discloses that the size of the regions between each of 
the plurality of segments is indicated in a manner such that the regions between with a 
greater number of frames are larger than the plurality of segments with a lesser number 
of frames (Figure 7). 

Referring to claim 6, Yeo discloses that the user selects one of the plurality of 
segments by interacting with the graphical user interface at a point within the displayed 
bounded spatial region associated with the selected one of the plurality of segments 
(Figure 1 and column 1 , lines 48-52), where the user selects the bounded region with 
the summary frames. 

Referring to claim 7, Yeo discloses that presentation of a selected one of the 
plurality of segments begins at the first frame of the segment irrespective of which point 
within the displayed bounded spatial region that the user interacted with (Figure 1 and 
column 1 , lines 48-52), with the user selecting a bounded region resulting in playing all 
the frames within this segment starting with the first frame to update the user on the 
video data. 

Referring to claim 9, Yeo discloses that presentation of a selected one of the 
plurality of segments begins at a frame of the segment temporally corresponding to the 
point within the displayed bounded spatial region that the user interacted with (Figure 1 
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and column 1, lines 48-52), where the video associated with selected segment is played 
and the beginning frame of this selected segment is played. This segment has a 
temporal position which conveys desired information to update the user's viewing. 

Referring to claim 10, Yeo discloses including a selector by which the user may 
alternatively select a chosen one of presentation of a selected one of the plurality of 
segments beginning at the first frame of the segment irrespective of which point within 
the displayed bounded spatial region that the user interacted with and presentation of a 
selected one of the plurality of segments beginning at a frame of the segment 
temporally corresponding to the point within the displayed bounded spatial region that 
the user interacted with (Figure 1 and column 1, lines 48-52). 

Referring to claim 11, Yeo, Tovinkere and Moran discloses claim 7 including a 
user-moveable scroll bar on said graphical user interface indicating the relative temporal 
location of currently-presented frames of said summary, wherein the user selects one of 
the plurality of segments by moving the scroll bar over the selected one of the plurality 
of segments, and where the scroll bar snaps to the beginning of the selected one of the 
plurality of segments (Moran, Figure 1 1 ). 

Referring to claims 12 and 39, Yeo, Tovinkere and Moran disclose at least two of 
the plurality of segments are temporally overlapping (Moran, Figure 1 1 ). 

Referring to claims 13 and 40, Yeo, Tovinkere and Moran disclose that the 
temporally overlapping segments are visually indicated in a manner such that each of 
the overlapping segments are independently identifiable (Moran, Figure 11). 
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Referring to claim 14, Yeo discloses that a user selects a portion of the video not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments (column 1 , line 65-column 2, line 1 ). In 
response to a user selecting to view a shot, a past segment that is associated with this 
selected shot is played to the user to ensure that the user knows the details of the 
video. 

Referring to claim 15, Yeo discloses that one of the plurality of segments is the 
segment most temporally adjacent to the portion of the video (column 1 , lines 53-57). 

Referring to claim 16, Yeo discloses that one of the plurality of segments is the 
next temporally related segment (column 2, lines 20-22). 

Referring to claim 1 7, Yeo discloses one of the plurality of segments is the 
previous temporally related segment (column 1, lines 53-57). 

Referring to claim 18, Yeo discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (Figure 1 and column 1, lines 48- 
52). 

Referring to claim 19, Yeo discloses that a user selects a portion of the video not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (Figure 1 , column 1 , 
lines 48-52 and column 1, line 65-column 2, line 1). 
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Referring to claim 20, Yeo discloses that a user selects a portion of the video not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments starting from the 
beginning thereof (Figure 1 , column 1 , lines 48-52 and column 1 , line 65-column 2, line 
1). 

Referring to claim 22, Yeo, Tovinkere and Moran disclose that the temporal 
information is hierarchical and is displayed in such a manner to retain a portion of its 
hierarchical structure (Moran, Figure 11). 

Referring to claim 23, Yeo, Tovinkere and Moran disclose that temporal 
information relates to overlapping time periods and the temporal information is 
displayed in such a manner to maintain the differentiation of the overlapping time 
periods (Moran, Figure 11). 

Referring to claim 24, Yeo, Tovinkere and Moran disclose that temporal 
information is displayed within a time line, wherein the temporal information is presented 
in a plurality of layers in a direction perpendicular to the length of the time line (Moran, 
Figure 11). 

Referring to claim 25, Yeo, Tovinkere and Moran disclose that the temporal 
information is displayed within a time line, wherein textual information is included within 
the time line (Moran, Figure 13). 
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Referring to claim 26, Yeo, Tovinkere and Moran disclose that the temporal 
information is displayed within a time line, wherein additional textual information is 
displayed upon selecting a portion of the time line (Moran, Figure 13). 

Referring to claim 27, Yeo, Tovinkere and Moran disclose that the temporal 
information is displayed together with a time line, wherein additional textual information 
is displayed together with selecting a portion of the time line (Moran, Figure 13). 

Referring to claim 28, Yeo, Tovinkere and Moran disclose that the temporal 
information is displayed within a time line, wherein additional audio annotation is 
presented upon presenting a portion of the time line (Moran, column 22, lines 1-5). 

Referring to claim 29, Yeo discloses a method of presenting information 
regarding a video comprising a plurality of frames (column 1, lines 6-10). Yeo discloses 
identifying a plurality of different segments of the video, where each of the segments 
includes a plurality of frames of the video (Figure 1 and column 1 , lines 6-1 0). The 
segments represent sequential frames of past and further video data associated with 
the currently playing video. Yeo discloses displaying, simultaneously with a segment of 
the video, a graphical user interface including information regarding the temporal 
location of the segment relative to at least one other of the segments of the video 
(Figure 1). Yeo discloses using a first visual indication and using a second visual 
indication different from said first visual indication to indicate different events within the 
video (Figure 7 and column 5, lines 13-21). The bounded region that is defined 
indicates video data as frames. This bounded region indicates the temporal relationship 
of the frames of the video with regions that visually indicate differences in the video 
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data. Yeo discloses displaying to a user at least one selector by which the user may 
interact with the interactive display to select for viewing selective identified ones of the 
plurality of segments (column 1 , lines 48-52). Yeo discloses receiving user selections of 
identified ones of the plurality of segments and presenting user selected ones of the 
plurality of different segments (column 1, lines 48-57). Selecting one segment results in 
the presentation of that video segment as opposed to another video segment. 

Yeo does not disclose a semantic characterization of an event. Tovinkere 
discloses detecting an event in a video characterized by a semantic event that includes 
a sports play (column 7, lines 40-43). It would have been obvious to one skilled in the 
art at the time of the invention to learn from Tovinkere a semantic characterization of an 
event. Tovinkere discloses that identifying the semantic event of a sports play ensures 
that the user has access to the most desired data within large amounts of video data. 
This provides motivation for Tovinkere to learn from Yeo. Therefore one skilled in the 
art at the time of the invention would have been motivated to learn from Tovinkere a 
semantic characterization of an event. 

Yeo and Tovinkere do not disclose displaying an interactive display of temporal 
location of the video as a plurality of segments. Moran discloses a graphical user 
interface with a plurality of segments (Figure 11). The plurality of segments has 
different visual indications that disclose temporal location for events within a video 
(Figure 11). It would have been obvious to one skilled in the art at the time of the 
invention to learn from Moran displaying an interactive display of temporal location of 
the video as a purality of segments. Yeo discloses a timeline structure with a plurality of 
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segments that visually indicate the different within the segments of the video data. The 
displaying of this timeline structure would allow the user to determine different types of 
events with the video data and access these events more easily and efficiently. This 
provides motivation for Yeo and Tovinkere to learn from Moran. Therefore one skilled in 
the art at the time of the invention would have been motivated to learn from Moran 
displaying an interactive display of temporal location of the video as a purality of 
segments. 

Referring to claim 30, Yeo discloses that the graphical user interface includes a 
generally rectangular region where each of the plurality of segments is indicated within 
the generally rectangular region (Figure 1). 

Referring to claim 31 , Yeo discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 7). 

Referring to claim 32, Yeo discloses that the size of the regions between each of 
the plurality of segments is indicated in a manner such that regions between with a 
greater number of frames are larger than the plurality of segments with a lesser number 
of frames (Figure 7). 

Referring to claim 33, Yeo, Tovinkere and Moran disclose an indicator that 
indicates the current position within the temporal information of a currently displayed 
portion of the video (Moran, column 27, lines 25-29). 



Application/Control Number: 10/016,941 Page 12 

Art Unit: 2173 

Referring to claim 34, Yeo, Tovinkere and Moran disclose that the indicator 
changes location relative to the temporal information as the portion of the currently 
displayed portion of video changes (Moran, column 27, lines 35-42). 

Referring to claim 36, Yeo, Tovinkere and Moran disclose indicating with an 
indicator the current position within the temporal information of a currently displayed 
portion of the video and modifying the position of the indicator within the temporal 
information which modifies the displayed portion of the video (Moran, column 27, lines 
35-42). 

Referring to claim 37, Yeo, Tovinkere and Moran disclose that the indicator is 
modified to a portion of the video that is not included within the plurality of segments 
(Moran, column 27, and lines 35-42). 

Referring to claim 41 , Yeo discloses that a user selects a portion of the video not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments (column 1 , line 65-column 2, line 1 ). 

Referring to claim 42, Yeo discloses that one of the plurality of segments is the 
segment most temporally adjacent to the portion of the video (column 1 , lines 53-57). 

Referring to claim 43, Yeo discloses that one of the plurality of segments is the 
next temporally related segment (column 2, lines 20-22). 

Referring to claim 44, Yeo discloses that one of the plurality of segments is the 
previous temporally related segment (column 1, lines 53-57). 

Referring to claim 45, Yeo discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
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presents the portion of the video from the start thereof (Figure 1 and column 1 , lines 48- 
52). 

Referring to claim 46, Yeo discloses that a user selects a portion of the video not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (Figure 1 , column 1 , 
lines 48-52 and column 1, line 65-column 2, line 1). 

Referring to claim 47, Yeo discloses that a user selects a portion of the video not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments starting from the 
beginning thereof (Figure 1 , column 1 , lines 48-52 and column 1 , line 65-column 2, line 
1). 

Referring to claims 49 and 70, Yeo, Tovinkere and Moran disclose that the 
temporal information is hierarchical and is displayed in such a manner to retain a portion 
of its hierarchical structure (Moran, Figure 11). 

Referring to claims 50 and 71 , Yeo, Tovinkere and Moran disclose that the 
temporal information relates to overlapping time periods and temporal information is 
displayed in such a manner to maintain the differentiation of the overlapping time 
periods (Moran, Figure 1 1 ). 
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Referring to claims 51 and 72, Yeo, Tovinkere and Moran disclose that the 
temporal information is displayed within a time line, wherein the temporal information is 
presented in a plurality of layers in a direction perpendicular to the length of the time line 
(Moran, Figure 11). 

Referring to claims 52 and 73, Yeo, Tovinkere and Moran disclose that the 
temporal information is displayed within a time line, wherein textual information is 
included within the time line (Moran, Figure 13). 

Referring to claims 53 and 74, Yeo, Tovinkere and Moran disclose that the 
temporal information is displayed within a time line, wherein additional textual 
information is displayed upon selecting a portion of the time line (Moran, Figure 13). 

Referring to claims 54 and 75, Yeo, Tovinkere and Moran disclose that the 
temporal information is displayed together with a time line, wherein additional textual 
information is displayed together with selecting a portion of the time line (Moran, Figure 
13). 

Referring to claims 55 and 76, Yeo, Tovinkere and Moran disclose that the 
temporal information is displayed within a time line, wherein additional audio annotation 
is presented upon presenting a portion of the time line (Moran, column 22, lines 1-5). 

Referring to claim 56, Yeo discloses a method of presenting information 
regarding audio identifying a plurality of different segments of the audio, where each of 
the segments includes a temporal duration of the audio (column 1, lines 45-62). The 
video information including audio is represented in segments, the segments represent 
temporal duration including further segments. Yeo discloses displaying simultaneously 
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with the segment of the audio a graphical user interface including information regarding 
the temporal location of the segment relative to at least one other of the segment of the 
audio (Figure 1 and column 1, lines 45-62). Yeo discloses using a first visual indication 
and using a second visual indication different from the first visual indication (Figure 7). 
Yeo discloses displaying to a user at least one selector by which the user may interact 
with the display to select for listening selective identified ones of the plurality of 
segments (column 1 , lines 46-56), the mouse pointer is the selector. Yeo discloses 
receiving user-selections of identified ones of the plurality of segments and presenting 
user-selected ones of the plurality of different segments (column 1 , lines 45-55). 

Yeo does not disclose a semantic characterization of an event. Tovinkere 
discloses detecting an event in a video characterized by a semantic event that includes 
a sports play (column 7, lines 40-43). It would have been obvious to one skilled in the 
art at the time of the Invention to learn from Tovinkere a semantic characterization of an 
event. Tovinkere discloses that identifying the semantic event of a sports play ensures 
that the user has access to the most desired data within large amounts of video data. 
This provides motivation for Tovinkere to learn from Yeo. Therefore one skilled in the 
art at the time of the Invention would have been motivated to learn from Tovinkere a 
semantic characterization of an event. 

Yeo and Tovinkere do not disclose displaying an interactive display of temporal 
location of the video as a plurality of segments. Moran discloses a graphical user 
interface with a plurality of segments (Figure 11). The plurality of segments has 
different visual indications that disclose temporal location for events within a video 



Application/Control Number: 10/016,941 Page 16 

Art Unit: 2173 

(Figure 11). It would have been obvious to one skilled in the art at the time of the 
invention to learn from Moran displaying an interactive display of temporal location of 
the video as a purality of segments. Yeo discloses a timeline structure with a plurality of 
segments that visually indicate the different within the segments of the video data. The 
displaying of this timeline structure would allow the user to determine different types of 
events with the video data and access these events more easily and efficiently. This 
provides motivation for Yeo and Tovinkere to learn from Moran. Therefore one skilled in 
the art at the time of the invention would have been motivated to learn from Moran 
displaying an interactive display of temporal location of the video as a purality of 
segments. 

Referring to claim 58, Yeo, Tovinkere and Moran disclose indicating with an 
indicator the current position within the temporal information of a currently displayed 

portion of the audio and modifying the position of the indicator within the temporal 
information which modifies the displayed portion of the audio (Moran, column 27, lines 
35-42). 

Referring to claim 59, Yeo, Tovinkere and Moran disclose that the indicator is 
modified to a portion of the audio that is not included within the plurality of segments 
(Moran, column 27, lines 35-42). 

Referring to claim 60, Yeo, Tovinkere and Moran disclose that at least two of the 
plurality of segments are temporally overlapping (Moran, Figure 1 1 ). 
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Referring to claim 61 , Yeo, Tovinkere and Moran disclose that the temporally 
overlapping segments are visually indicated in a manner such that each of the 
overlapping segments are independently identifiable (Moran, Figure 11). 

Referring to claim 62, Yeo discloses that a user selects a portion of the audio not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments (column 1 , line 65-column 2, line 1 ). 

Referring to claim 63, Yeo discloses that one of the plurality of segments is the 
segment most temporally adjacent to the portion of the audio (column 1 , lines 53-57). 

Referring to claim 64, Yeo discloses that one of the plurality of segments is the 
next temporally related segment (column 2, lines 20-22). 

Referring to claim 65, Yeo discloses that one of the plurality of segments is the 
previous temporally related segment (column 1, lines 53-57). 

Referring to claim 66, Yeo discloses that a user selects a portion of the audio 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio from the start thereof (Figure 1 and column 1 , lines 48- 
52). 

Referring to claim 67, Yeo discloses that a user selects a portion of the audio not 
included within the plurality of segments, wherein in response thereto, the system 
presents on the plurality of segments, wherein in response thereto, the system presents 
the portion of the audio within the plurality of segments (Figure 1 , column 1 , lines 48-52 
and column 1, line 65-column 2, line 1). 
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Referring to claim 68, Yeo discloses that a user selects a portion of the audio not 
included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio within the plurality of segments starting from the 
beginning thereof (Figure 1 , column 1 , lines 48-52 and column 1 , line 65-column 2, line 
1). 

Referring to claim 77, Yeo discloses that a user selectable skip function skips a 
set of frames to a modified location of the video in at least one of a forward temporal 
direction or a reverse temporal direction, and displays the video at the modified location 
(column 1, lines 46-57). The user selectable segments represents skip function that 
allows the user to jump to a modified location of the video. 

Referring to claim 78, Yeo discloses that a user selectable skip function skips to 
a later temporal segment or a previous temporal segment, and displays the video at the 
later temporal segment or the previous temporal segment, respectively (column 1 , lines 
46-57). 

Referring to claim 79, Yeo discloses that a user selectable scan function skips a 
set of frames to a modified location of the video in at least one of a forward temporal 
direction or a reverse temporal direction, and displays the video at the modified location, 
and thereafter automatically skips another set of frames to another modified location of 
the video in at least one of the forward temporal direction or the reverse temporal 
direction, and displays the video at another modified location (column 1 , lines 46-57). 
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The segments allow for the user to skip to modified video locations as each segment is 
selected. 

Referring to claim 80, Yeo discloses that at least one of the forward temporal 
direction and reverse temporal direction are consistent with different segments (column 
1, lines 46-57). 

Referring to claim 81 , Yeo discloses that display of the video is at the start of the 
respective one of the different segments (column 1, lines 46-57), selecting the different 
segment allows for the user to view from the beginning. 

Referring to claim 82, Yeo discloses that display of the video is at a 
predetermined offset within the respective one of the different segments (column 1 , 
lines 46-57). 

Referring to claim 83, Yeo discloses that the graphical user interface displays a 
respective image associated with at least a plurality of the different segments (Figure 1). 

Referring to claim 84, Yeo, Tovinkere and Moran discloses that respective image 
associated with the currently presented different segments is visually highlighted 
(Moran, column 23, lines 45-50). 

Referring to claim 85, Yeo, Tovinkere and Moran discloses that during 
presentation of the video the visually highlighted respective images are highlighted in a 
substantially regular interval while the sequence of presentation of the video is at 
substantially irregular intervals (Moran, column 23, lines 45-50). 

Referring to claim 86, Yeo discloses that the presentation of the different 
segments may be modified by a plurality of different functions, and wherein the user 
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may customize anotlier function, not previously explicitly provided, by combining a 
plurality of the plurality of different functions into a single function (column 1 , lines 45- 
65). 

Allowable Subject Matter 

3. Claims 21 , 48 and 69 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

4. The following is a statement of reasons for the indication of allowable subject 
matter: With respect to independent claims 21 , 48, 69 and independent claim 87, the 
combination of Yeo, Tovinkere and Moran disclose summarizing video data into frames 
and displaying this data. There are plurality of segments that are displayed based on 
distinct events that are associated with sports plays. Prior art combinations do not 
properly teach the a distinct pattern where in response to the user selecting a portion of 
the video not included within the plurality of segments, the portion not included in the 
segments is displayed. Furthermore, after this displaying the system goes back to 
displaying only the segments in temporal order without the portions of the video not 
included within the segments. Prior arts disclosed do not disclose in addition to the 
features of the independent claims in combination with the features of claims 21 , 48 and 
69. 

5. Claim 87 is allowed. 

Response to Arguments 
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6. Applicant's arguments filed 1/29/09 have been fully considered but they are not 
persuasive. 

Applicant argues that the references do not disclose displaying to a user the 
relative location for a first semantic characterization of the sports play in the video using 
a first visual indication and displaying the relative location for a second semantic 
characterization of the sports play in a video using a second visual different from the 
first visual indication on a second portion. Figure 1 1 of Moran discloses a user interface 
with different types of media data that are identified based on content. The combination 
of Yeo, Tovinkere and Moran disclose the above feature. Moran discloses how there 
are distinct areas in which different types of information are displayed and accessible to 
the user. Furthermore the combination is proper. The references cited disclose video 
interface elements that provide data to the user with information that is valuable and 
associated with the video being viewed. 

Conclusion 

7. Responses to this action should be submitted as per the options cited below: The 
United States Patent and Trademark Office requires most patent related 
correspondence to be: a) faxed to the Central Fax number (571-273-8300) b) hand 
carried or delivered to the Customer Service Window (located at the Randolph Building, 
401 Dulany Street, Alexandria, VA 22314), c) mailed to the mailing address set forth in 
37 CFR 1 .1 (e.g., P.O. Box 1450, Alexandria, VA 22313-1450), or d) transmitted to the 
Office using the Office's Electronic Filing System. 
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Any inquiry concerning tliis communication or earlier communications from the 
examiner should be directed to NAMITHA PILLAI whose telephone number is (571 ) 
272-4054. The examiner can normally be reached from 9:00 AM - 5:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, Kieu Vu can be 
reached on (571 ) 272-4057. 

All Internet e-mail communications will be made of record in the application file. 
PTO employees do not engage in Internet communications where there exists a 
possibility that sensitive information could be identified or exchanged unless the record 
includes a properly signed express waiver of the confidentiality requirements of 35 
U.S.C. 122. This is more clearly set forth in the Interim Internet Usage Policy published 
in the Official Gazette of the Patent and Trademark on February 25, 1997 at 1 195 OG 
89. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Group receptionist whose telephone number is 
(571)272-2100. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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